In order to enhance the robustness of the visual question answering model, a bias reduction method is proposed. Based on this, the influence of language and visual information on bias effect is explored. Furthermore, two bias learning branches are constructed to capture the language bias, and the bias caused by both language and images. Then, more robust prediction results are obtained by using the bias reduction method. Finally, based on the difference in prediction probabilities between standard visual question answering and bias branches, samples are dynamically weighted, allowing the model to adjust learning levels for samples with different levels of bias. Experiments on VQA-CP v2.0 and other data sets demonstrate the effectiveness of the proposed method and alleviate the influence of bias on the model.

The accuracy of the existing word-based domain feature learning methods on domain discrimination is still low and the further research for domain feature learning is required. In order to improve domain discrimination and provide accurate translation, this paper proposes a word-based domain feature-sensitive learning mechanism, including 1) the context feature encoding at encoder side, to widen the study range of word-based domain features, introducing convolutional neural networks (CNN) in encoder for extracting features from word strings with different lengths in parallel as word context features; and 2) enhanced domain feature learning. A domain discriminator module based on multi-layer perceptions (MLP) is designed to enhance the learning ability of obtaining more accurate domain proportion from word context features and improve the accuracy of word domain discrimination. Experiments on English-Chinese task of UM-Corpus and English-French task of OPUS show that the average BLEU scores of the proposed method exceed the strong baseline by 0.82 and 1.06 respectively. The accuracy of domain discrimination is improved by 10.07% and 18.06% compared with the baseline. More studies illustrate that the improvements of average BLEU scores and accuracy of domain discrimation are contributed by the proposed word-based domain feature-sensitive learning mechanism.

Currently, neural machine translation (NMT) is insufficient in capturing the semantic and structural relationships between clauses in complex sentences, which often results in poor discourse coherence of long and complex sentence translation. To address this problem, the paper proposes a Chinese-English NMT approach by integrating the clause alignment knowledge into NMT. Firstly, a labeling scheme combining manual and automatic annotation is introduced to annotate a large-scale clause aligned Chinese-English parallel corpus that provides rich clause-level Chinese-English bilingual alignment knowledge for model training. Then, a NMT model is designed based on clause alignment learning for enhancing the ability of the model to learn the semantic structure relationships between clauses within complex sentences. Experimental results on WMT17, WMT18 and WMT19 Chinese-English translation tasks demonstrate that proposed method can significantly improve the NMT performance. Evaluation and analysis show that proposed method can effectively improve the discourse coherence of complex sentence in Chinese-English machine translation.

In multi-modality scenarios such as commodity descriptions and news comments, existing paraphrase generation models can not utilize information from image and therefore result in the loss of semantics in the generated paraphrases. In order to solve this problem, this paper first propose the Multi-modality Paraphrase Generation (MPG) model to integrate image information for paraphrase generation. In MPG, in order to integrate the image information corresponding to the original sentence, the authors first construct an abstract scene graph and transform the image features into node features of the scene graph. Furthermore, the constructed scene graph was utilized to generate paraphrase, by using the relational graph convolutional neural network for encoder and graph-based attention mechanism for decoder. In the evaluation stage, a sentence pair similarity calculation method was proposed to select sentence pairs describing same objects from the MSCOCO data set, and then evaluation experiments were conducted. Experimental results show that the proposed MPG model achieve better semantic fidelity, which indicates that the integration of image information is effective in improving the quality of the paraphrase generation in multi-modality scenarios.

The authors introduce the concept and steps of entity linking in detail, and the problems and research status of named entity linking based on deep learning in recent years, analyze the problems and corresponding solution models of entity linking and present related data sets and evaluation methods. The authors summarize the current status of entity linking in international evaluation conferences and analyze the future research directions.

Prior work on adversarial example generation with syntactically controlled paraphrase networks requires large-scale paraphrase parallel corpora to train models. The performance of the model is seriously limited by the domain and scale of paraphrase parallel corpus. To solve this problem, this paper proposes an unsuprervised syntactically controlled paraphrase model to generate adversarial examples which only needs monolingual data. Specifically, variational autoencoder is used to learn model, which maps a sentence and a syntactic parse tree into semantic and syntactic variables, respectively. By learning to reconstruct the input sentence from syntactic and semantic variables, the model effectively learns to generate syntactic paraphrases without using any parallel data. Experiment results on unsupervised sentence paraphrasing and adversarial example generation demonstrate that the proposed model achieves new state-of-the-art results on unsupervised paraphrase generation and generate effective adversarial examples. These examples can be used to improve the robustness and generalization of NLP (natural language processing) model.

Focusing on the specific information of the positional relationship between objects in the image, a neural network image summary generation model integrating spatial relationship mechanism is proposed, in order to provide key information (object position or trajectory) for downstream tasks such as visual question answering and voice navigation. In order to enhance the learning ability of the positional relationship between objects of the image encoder, the geometric attention mechanism is introduced by improving the Transformer structure, and the positional relationship between objects is explicitly integrated into the appearance information of the objects. In order to assist in the completion of specific information-oriented extraction and summary generation tasks, a data production method for relative position relations is further proposed, and the image abstract data set Re-Position of the position relations between objects is produced based on the SpatialSense data set. The experimental results of comparative evaluation with five typical models show that the five indicators of the proposed model are better than those of other models on the public test set COCO, and all six indicators are better than those of other models on Re-Position data set.

Taking Chinese as the research object, the authors put forward the method to construct large-scale and high-quality paraphrase parallel corpora. The paraphrase data augmentation method include transfering English paraphrase corpus to Chinese, by using the method of translation engines, and manually annotating evaluation data set. Based on the constructed Chinese paraphrase data, the validity of the paraphrase data construction application method is verified in the paraphrase recognition task and natural language inference task. Firstly, the paraphrase recognition data is generated based on the constructed paraphrase corpus, and the attention-based neural network model of sentence matching is pre-trained to capture the paraphrase information. Then, the pre-trained model is applied to the natural language inference task to improve the performance. The experimental results on the open set show that the constructed paraphrase corpus can be effectively applied to the paraphrase recognition task, and the model can learn paraphrase knowledge. When applied to natural language inference task, paraphrase knowledge can effectively improve the accuracy of natural language inference models and verify the effectiveness of paraphrase knowledge for downstream semantic understanding tasks. Meanwhile, the proposed construction method for the paraphrase corpus is language-independent, which can provide more training data for other languages and fields, generate high-quality paraphrase data, and further improve the performance of other tasks.

Neural network encoder-decoder framework has become the popular method for paraphrase generation, but there are still two problems. On the one hand, there are such issues as inaccurate entity words, unknown words and word repetition in the generated paraphrase sentences. To solve the first problem, we proposed a multimechanism fused paraphrase generation model to improve the decoder. The copy mechanism was used to copy words form input sentence for improving the generation of entity and unknown words. The coverage mechanism was used to model historical attention information to avoid word repetition. On the other hand, the limited-scale parallel paraphrase corpus limits the learning ability of the encoder. We proposed to jointly learn auto-encoding task, which shares one encoder with paraphrase generation task. The joint auto-encoding task enhances the learning ability of the encoder. Experimental results on Quora paraphrase dataset show that the multi-mechanism fused paraphrase generation model with joint auto-encoding task can effectively improve the performance of paraphrase generation.

Paraphrase identification involves natural language semantic understanding. Most previous methods regarded sentences as sequential structures, and used sequential neural network for semantic composition. These methods do not consider the influence of syntactic structure on semantic computation. In this paper, we proposed a neural paraphrase identification model based on syntactic structure, and designed a tree-based neural network model for semantic composition, which extended the semantic representation from word level to phrase level. Furthermore, this paper proposed a syntactic tree alignment mechanism based on phrase-level semantic representation, and extracted features by using cross-sentence attention mechanism. Finally, a self-attention mechanism was used to enhance semantic representation, which could effectively model context information based on syntactic structure. Experiments on Quora paraphrase dataset show that the performance of paraphrase identification has been improved to 89.3% accuracy. The results further prove that the proposed semantic composition method based on syntactic structure, phrase-level cross sentence attention and self-attention are effective in improving paraphrase identification.

A piecewise convolutional neural network with gating mechanism is proposed, which would automatically filter positive correlation features at word-level. Moreover, the idea of soft-label is introduced to the gating mechanism to weaken the impact of hard labels on noise filtering. Combined with sentence-level noise filtering, the overall performance of the model is improved. The experimental results on the public dataset show that the proposed model has a significant improvement compared to the sentence-level noise filtering methods.